

Bl 



(19) 



Hff 



ropaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(43) Date of publication: 

28.08.1996 Bulletini 996/35 



(11) EP 0 729 278 A2 

EUROPEAN PATENT APPLICATION 

(51) Int. Cl.^: H04N9/04 



(21) Application number: 96102856.0 

(22) Date of filing: 26.02,1996 



(84) Designated Contracting States: 
DE FR GB 


(72) Inventor: Okada. Hidefumi 


Daito-shi, Osaka (JP) 


(30) Priority: 27.02.1995 J P 38632/95 


(74) Representative: Glawe, Delfs, Moll & Partner 


(71) Applicant: SANYO ELECTRIC Co., Ltd. 
Moriguchi-shi, Osaka 570 (JP) 


Patentanwalte 
Postfach 26 01 62 
80058 Munchen (DE) 



(54) 



nrSL^^S' ^ ""fT"^ generating inter-pixel color signal component by Interpolating 
primary color signals from neighboring pixels h ^ y 

(57) A 1 -chip color video camera providing such a 
frequency characteristic that suffers from small attenua- 
tion up to high frequency range for a G signal which has 
highest degree of contribution to brightness, and hence 
providing high resolution, is disclosed. In the 1-chip 
color video camera (100) having a color separation cir- 
cuit (8) for processing signals obtained from a solid- 
state image sensor (l) in which color filters (85) of R. G 

FIG. 8 



and B primaries arranged mosaic-wise for respective 
pixels, color signal components at a central portion of a 
pixel block consisting of four pixels of two rows by two 
columns on the solid-state image sensor are generated 
by interpolation using color signal components of a plu- 
rality of neighboring pixels. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to a color video camera. More specifically, the present invention relates to a 1-chip 
color video camera having a color separation circuit for processing signals from a solid-state image sensor in which 
color filters of primaries, that is. red. green and blue (hereinafter simply referred to as R. G and B) are arranged mosaic- 
wise. 

Description of the Background Art 

As disclosed in the Description of the Prior Art of Japanese Patent Uying-Open No. 63-97078 (H04N9/N7), a color 
video camera using 1-chip for primaries includes a solid-state image sensor having a photoreceptor portion, a charge 
transfer portion and a transfer control portion. On the photoreceptor portion, a mosaic microf ilter is deposited. 

More specrtically, referring to Fig. 1 , the photoreceptor portion 85 of solid-state image sensor 1 includes a plurality 
of photoelectric converting elements arranged in a matrix, and mosaic elements of the microfilter are correspondingly 
arranged for respective photoelectric converting elements. Namely, a microfilter corresponding to one color, for exam- 
ple, one of R. G and B is allotted to one photoelectric converting element. There are various combinations of filter 
arrangement of R. G and B in such a mosaic microfilter. One typical example has a combination in which odd numbered 
rows viewed from below includes GRGR .... and even numbered rows include GBGB .... as shown in Fig. 2. More spe- 
cifically, filters corresponding to green, which requires high resolution, are arranged in black squares of a checker 
board, and R and B filters are arranged on white squares (hereinafter, referred to as checkenA^ise arrangement). In this 
case, rows including R filters and rows including B filters are arranged alternately This arrangement is generally 
referred to Bayer type arrangement. 

From respective pixels of the solid-state image sensor on which the microfilter is arranged as described above, 
color signals corresponding to the colors of associated color filters are output. The color signals are separated into R, 
G and B color signals respectively, by a color separation circuit in a succeeding stage. 

Fig. 1 is a schematic block diagram showing a structure of the solid-state image sensor, that is, a charge coupled 
device (hereinafter referred to as CCD) 1. 

CCD! includes a photoreceptor portion 85 having a plurality of photoelectric converting elements, for example, 
photodtodes. arranged in a matrix corresponding to respective pixels; a plurality of vertical transfer registers 83 receiv- 
ing charges stored in the diodes corresponding to the incident light for transferring charges successively in vertical 
direction; a vertical driving circuit 81 for outputting a clock pulse voltage for controlling the operation of vertical transfer 
register 83; a horizontal transfer register 84 receiving charges transferred by respective charge transfer registers for 
honzontally transferring and outputting signals by converting the successively transferred signal charges to a voltage; 
and a horizontal driving circuit 82 for outputting a clock voltage for controlling the operation of the horizontal transfer reg- 
ister 84. 

In other words, solid-state image sensor 1 has a structure of a so-called interline transfer CCD. 

Therefore, in each pixel at photoreceptor portion 85. light intensity of incident light received through the correspond- 
ing microfilter is converted to an electrical signal, and output as a corresponding analog signal, row by row of pixels. 

When the mosaic microfilter such as described above is used, only an R signal is obtained from the pixel on which 
an R filter is disposed, and G and B signals cannot be obtained. Therefore, the G and B signals of this pixel must be 
generated by interpolation from G and B signals of neighboring pixels. 

In a so-called digital camera in which signals from a solid-state image sensors are digitized for further processing, 
interpolation of a missing signal has been performed by the following operation. 

More specifically, in accordance with the arrangement of color filters, a two-dimensional missing signal interpolating 
digital filter performs weighting, using a weight coefficient predetermined for each pixel. More specifically, signals of the 
same color as the missing signal obtained from adjacent neighboring signals are multiplied by respective weight coef- 
ficients, and resulting multiplied values are added and then divided by a sum of all weight coefficients, that is. a so- 
called weighted means is calculated, so as to obtain a color signal which is of the same color as the missing signal. 

Fig. 2 shows a pattern of arrangement of R. G and B of the mosaic microfilter of CCD1 shown in Fig. 1 . In the pat- 
tern shown in Fig. 2. there are four possible arrangements, that is. HI, H2. H3 and H4 shown in Figs. 3A. 4A. 5A and 
6A. respectively of color filters for a block of three pixels by three pixels with an arbitrary pixel positioned at the center. 

Fig. 3 A shows one (hereinafter referred to as arrangement H 1 ) of four arrangements, in which a G filter is deposited 
on the central pixel. In this case, the G signal obtained from this pixel is multiplied by the weight coefficient of "4" as 
shown in Fig. 38, and then it is divided by "4". so that the G signal is used as it is. as the G signal of the central pixel. 
As for the R signal. R signals obtained from upper and lower adjacent pixels on which R filters are deposited are multi- 
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plied by the weight coefficient of "2" respectively, as shown in Fig. 3C. and the value obtained by adding the R signals 
of the upper and lower pixels is divided by -4". whereby the R signal of the central pixel is generated. Further, as for the 
8 signal. B signals obtained from left and right adjacent pixels on which B filters are arranged are multiplied by the 
weight coefficient of "2". respectively, as shown in Fig. 3D, and the value obtained by adding the B signals of the left and 
rtght pixels is divided by "4". whereby the B signal for the central pixel is generated. 

Fig. 4(A) shows another (referred to as arrangement H2) of the four arrangements, in which a B filter is deposited 
on the central pixel. Therefore, as for the G signal, G signals obtained from upper, lower, left and right four pixels are 
multiplied by the weight coefficient of "1" as shown in Fig. 4B. and the value obtained by adding the G signals from these 
four pixels ts divided by "4'\ Thus the G signal for the central pixel is generated. As for the R signal. R signals obtained 
from upper left, upper right, lower left and lower right four pixels are multiplied by the weight coefficient of "1". respec- 
tively, as shown in Fig. 4C. and the value obtained by adding the R signals from these four pixels is divided by "4" 
whereby the R signal for the central pixel is generated. As for the B signal, since a B filter is disposed on the central 
pixel, the B signal obtained from this pixel is multiplied by the weight coefficient of "4" as shown in Fig 4D and the 
resulting value is divided by "4". so that the B signal is used, as it is. as the B signal of the central pixel. 

Fig. 5A shows a still another one (referred to as arrangement 3) of the four arrangements, in which an R filter is 
disposed on the central pixel. Therefore, as for the G signal. G signals obtained from upper, lower left and right four 
pixels are multiplied by the weight coefficient of "1 " as shown in Rg. 58. and the value obtained by adding the G signals 
frorn these four pixels is divided by "4". whereby G signal of the central pixel is generated. As for the R signal, since an 
R filter IS disposed on the central pixel, the R signal obtained from this pixel is multiplied by the weight coefficient of "4" 
as shown m Fig. 5C, and the resulting value is divided by "4". Thus the R signal is used, as it is, as the R signal for the 
central pixel. As for the B signal. B signals obtained from upper left, upper right, lower left and lower right four pixels are 
multiplied by the weight coefficient of "1 respectively, as shown in Fig. 5D, and the value obtained by adding the B sig- 
nals from these four pixels is divided by "4". whereby the B signal for the central pixel is generated. 

Fig. 6A shows still another one (referred to as arrangement H4) of the four arrangements, in which a G filter is dis- 
posed on the central pixel. As shown in Fig. 6B. the G signal obtained from the central pixel is multiplied by the weight 
coefficient of "4". and the resulting value is divided by "4". so that the G signal, as it is. is used as the G signal for the 
central pixel. As for the R signal, R signals obtained from left and right adjacent pixels on which R filters are disposed 
are multiplied by the weight coefficient of "2". respectively, as shown in Fig. 6C. and the value obtained by adding these 
R signals of the left and right pixels is divided by 4. whereby the R signal for the central pixel is generated. As for the B 
signal, B signals obtained from upper and lower adjacent pixels on which B filters are disposed are multiplied by the 
weight coefficient of "2". respectively, as shown in Fig. 6D. and the value obtained by adding the B signals of the upper 
and lower pixels is divided by "4". whereby the B signal for the central pixel is generated. 

Such interpolation of color signals as described above is performed by means of an interpolation digital filter con- 
sisting of a two-dimensional FIR filter (Finite Impulse Response) filter. 

The transfer function H (Z) of the FIR filter with respect to the aforementioned weight coefficients are as follows. 

[ARRANGEMENT H1] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H(z)=Ihmz''"=1 

(HORIZONTAL DIRECTION OF R SIGNAL) 

H (z) = 1 

(VERTICAL DIRECTION OF R SIGNAL) 

H(2)=Lhmz'^=1x2-%0xz-Vlx2-2=1+z-2 
(HORIZONTAL DIRECTION OF B SIGNAL) 



55 (VERTICAL DIRECTION OF B SIGNAL) 



H (z)=1+z 



Z{z) 
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[ARRANGEMENT 2] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 
5 H (2)=!:hmz "^=1 xz -°+2x2 ' Vl xz -^ = 1+22 •''+2 '2 

(HORIZONTAL & VERTICAL DIRECTIONS OF R SIGNAL) 

H (z)=Ehm2 '"^ = 1 X2 -^Oxz "Vl xz '^1 +2 

10 

(HORIZONTAL & VERTICAL DIRECTIONS OF B SIGNAL) 

H (2) = 1 

15 [ARRANGEMENTS] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H(2)=Ehm2-"^=1xz^+2x2-''+1xz'^=1+22'"'+z-2 

20 

(HORIZONTAL & VERTICAL DIRECTIONS OF R SIGNAL) 

H (z) = 1 

25 (HORIZONTAL & VERTICAL DIRECTIONS OF B SIGNAL) 

H(2)= 1 + 2-2 

[ARRANGEMENT 4] 

30 

(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H (2) = 1 

35 (HORIZONTAL DIRECTION OF R SIGNAL) 

H (2) = 1 + 2 -2 

(VERTICAL DIRECTION OF R SIGNAL) 

40 

H (2) = 1 

(HORIZONTAL DIRECTION OF B SIGNAL) 

H (z) = 1 

(VERTICAL DIRECTION OF B SIGNAL) 

H(z) = 1 +2*^ 

50 

Fig. 7 is a graph showing characteristics of the interpolation filter as represented by the transfer functions above, in 
which the ordinate indicates gain of the interpolation filter, while the abscissa represents operational frequency of the 
interpolation filter. More specifically, for the image pick up signal sampled at a prescribed sampling time, the frequency 
plotted on the abscissa corresponds to a reciprocal of the period of spatial change in the picked-up image. 
55 The characteristics of the interpolation filter as represented by the transfer functions above for each of the R. G and 
B color signals correspond to the curve P1 . P2 and P3 shown in Fig. 7 with respect to the horizontal and vertical direc- 
tions. 
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In summary, the present invention provides a 1-chip color video camera including a solid-state image sensor and 
an interpolating circuit. 

The solid-state image sensor includes photoelectric converting elements, corresponding to respective pixels, 
arranged in an array. The solid-state image sensor includes a color filter array in which color filters of primaries are dis- 
posed in a prescribed arrangement, corresponding to the photoelectric converting elements, on a photoreceptor sur- 
face side. The interpolating circuit receives an output from the solid-state image sensor and outputs a corresponding 
color signal, and it includes a parallel color signal output circuit, a control circuit, and a color separation circuit. The par- 
allel color signal output circuit receives output from the solid-state image sensor, and successively outputs color signal 
corresponding to a prescribed even number of rows of pixels column by column in parallel and in synchronization. The 
control circuit outputs a synchronized interpolation designating signal in accordance wrth correspondence between the 
color signals output column by column from the parallel color signal output circuit and the arrangement of predeter- 
mined weight coefficients for the prescribed arrangement of the color filter array. The color separation circuit receives 
outputs from the parallel color signal output circuit and. in accordance with the interpolation designating signal, per- 
forms interpolation of color signals from a pixel block including a prescribed even numbered rows and a prescribed even 
numbered columns of pixels, and outputs successively and in synchronization, a color signal corresponding to the cen- 
tral position of the pixel block. 

In accordance with another aspect, the 1 -chip color video camera includes a solid-state image sensor and an inter- 
polating circuit. 

The solid-state image sensor includes photoelectric converting elements corresponding to respective pixels, 
arranged in an array. The solid-state image sensor includes a color filter array in which color filters of primaries are 
arranged mosaic-wise corresponding to the photoelectric converting elements, on a photoreceptor surface side. The 
color filter array includes, in a color filter arrangement of arbitrary two rows by two columns, green filters arranged in 
diagonal direction. The interpolating circuit receives an output from the solid-state image sensor and interpolates green 
signal component at the central portion of a pixel block corresponding to the arbitrary color filter arrangement of two 
rows by two columns, by mean value of green signals obtained from photoelectric converting elements corresponding 
to the green filters arranged in the diagonal direction. 

According to a still further aspect of the present invention, the 1-chip color video camera includes a solid-state 
image sensor and an interpolating circuit. The solid-state image sensor includes photoelectric converting elements, 
corresponding to respective pixels, arranged in an an-ay. The solid-state image sensor includes a color filter array in 
which color filters of primaries are arranged mosaic-wise corresponding to the photoelectric converting elements, on a 
photoreceptor surface side. The color filter array includes red and blue filters arranged checkenwise. and rows including 
red filters and rows including blue filters are arranged alternately. The interpolation circuit interpolates each of red and 
blue color signal components at the central portion of a pixel block, based on the signals from the pixel block including 
4 rows by 4 columns of pixels, that is, 16 pixels. The. interpolating circuit includes a two-dimensional non-recursive dig- 
ital filter circuit for switching column-wise values of vertical sums of weight coefficients between (0, 3. 0. 1) and (1 , 0, 3, 
0), in accordance with the arrangement of color filters corresponding to the sixteenth pixels. 

According to a still further aspect of the present invention, the 1-chip color video camera includes a solid-state 
image sensor and an interpolating circuit. 

The solid-state image sensor includes a color filter array in which color filters of primaries are arranged mosaic- 
wise corresponding to photoelectric converting elements on a photoreceptor surface side. The interpolating circuit gen- 
erates a plurality of color signal components at a position shifted by half a pixel in horizontal and vertical directions from 
the center of an arbitrary pixel, based on color signal component of a plurality of neighboring pixels. 

Therefore, an advantage of the present invention is that such a frequency characteristic that suffers from little atten- 
uation up to the high frequency range is provided for the G signal which contributes most to brightness, and hence high 
resolution becomes possible. 

Another advantage of the present invention is that generation of a color false signal is suppressed, since difference 
between frequency characteristics of R and B signals and that of G signal is small at 1/2 Nyquist frequency. 

A still another advantage of the present invention is that same frequency characteristics can be provided for every 
pixel with respect to each of the three primaries. 

The foregoing and other objects, features, aspects and advantages of the present invention will become more 
apparent from the following detailed description of the present invention when taken in conjunction with the accompa- 
nying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic block diagram showing a structure of a conventional CCD1 . 
Fig. 2 is a schematic diagram showing a structure of a mosaic filter disposed on a conventional CGD1 . 
Figs. 3A to 3D are illustrations showing allotment of weight coefficients for interpolation in a conventional color 
video camera, in which Fig. 3A shows filter arrangement. Fig. 3B shows an arrangement of weight coefficients for the 
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G signal. Fig. 3C shows an arrangement of weight coefficients for the R signal and Fig. 3D shows an arrangement of 
weight coefficients for the B signal. 

Figs. 4A to 4D are illustrations showing allotment of weight coefficients for interpolation in a conventional color 
video camera, in which Fig. 4A shows filter arrangement. Fig. 4B shows an arrangement of weight coefficients for the 
G signal. Fig. 4C shows an arrangement of weight coefficients for the R signal and Fig. 4D shows an an-angement of 
weight coefficients for the B signal. 

Figs. 5A to 5D are illustrations showing allotment of weight coefficients for interpolation in a conventional color 
video camera, in which Fig. 5A shows filter arrangement. Fig. 58 shows an arrangement of weight coefficients for the 
G signal. Fig. 5C shows an arrangement of weight coefficients for the R signa! and Fig. 5D shows an arrangement of 
weight coefficients for the B signal. 

Figs. 6A to 6D are illustrations showing allotment of weight coefficients for interpolation in a conventional color 
video camera, in which Fig. 6A shows filter arrangement. Fig. 6B shows an arrangement of weight coefficients for the 
G signal. Fig. 6C shows an arrangement of weight coefficients for the R signal and Fig. 6D shows an an-angement of 
weight coefficients for the B signal. 

Fig. 7 is a graph showing frequency characteristics of three primary signals after interpolation in the conventional 
color video camera. 

Fig. 8 is a block diagram showing a structure of a color signal processing circuit 100 in accordance with one embod- 
iment of the present invention. 

Fig. 9 shows a position of interpolation portion in a pixel block in accordance with one embodiment of the present 
invention. 

Figs. 10A to 100 are illustrations showing allotment of weight coefficients for arrangement 1 for interpolation in 
accordance with one embodiment of the present invention in which Fig. 10A shows filter arrangement. Fig. 10B shows 
arrangement of weight coefficients for the G signal. Fig. IOC shows arrangement of weight coefficients for the R signal, 
and Fig. lOD shows arrangement of weight coefficients for the B signal. 

Figs. 11A to 11D are illustrations showing allotment of weight coefficients for arrangement 2 for interpolation in 
accordance with one embodiment of the present invention in which Fig. 1 1 A shows filter arrangement, Fig. 1 1B shows 
arrangement of weight coefficients for the G signal. Fig, 1 1C shows arrangement of weight coefficients for the R signal, 
and Fig. 1 1 D shows arrangement of weight coefficients for the B signal. 

Figs. 12A to 120 are illustrations showing allotment of weight coefficients for arrangement 3 for interpolation in 
accordance with one embodiment of the present invention in which Fig. 12A shows filter arrangement, Fig. 1 2B shows 
arrangement of weight coefficients for the G signal. Fig. 12C shows arrangement of weight coefficients for the R signal, 
and Fig. 120 shows arrangement of weight coefficients for the B signal. 

Figs. 13A to 130 are illustrations showing allotment of weight coefficients for arrangement 4 for interpolation in 
accordance with one embodiment of the present invention in which Fig. ISA shows filter an-angement, Fig. 13B shows 
arrangement of weight coefficients for the G signal. Fig. 13C shows arrangement of weight coefficients for the R signal, 
and Fig. 130 shows arrangement of weight coefficients for the B signal. 

Fig. 14 shows position of interpolation portion on the CCD in accordance with one embodiment of the present 
invention. 

Fig. 1 5 shows frequency characteristics of three primary signals in accordance with one embodiment of the present 
invention. 

Fig. 16 shows switching control of a selector circuit in a color signal processing circuit 100 in accordance with one 
embodiment of the present invention. 

^ DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 8 is a block diagram showing a structure of a color signal processing circuit 100 including components from a 
CCDi as an image sensor up to a color separation circuit 8, in a color video camera! in accordance with one embodi- 
ment of the present invention. 

The incident light is formed on CCDi by means of a lens (not shown), and photo-electrically converted to an image 
signal. A microfilter 70 including R. G and B color filters arranged mosaic-wise is provided on the photoreceptor surface 
of CCDI . It is assumed that the arrangement of respective color filters of mosaic microfilter 70 is the same as the prior 
art example shown in Fig. 2. The light which has passed through the lens is supplied through microfilter 70 to photore- 
ceptor portion of CCOl. In accordance with the intensity of incident light received through the filter, charges stored in 
photoreceptor portion 85 for one field period are transferred by vertical transfer register 83 and horizontal transfer reg- 
ister 84 to be output externally from CCDi . 

More specifically. CCDI includes photoreceptor portion 85; vertical transfer register 83 tor vertically transferring an 
output in accordance with the light intensity received at photoreceptor portion 85; horizontal transfer register 84 
arranged at a terminating end of vertical transfer register for transferring charges transferred from vertical transfer reg- 
ister in horizontal direction; a vertical driving circuit 81 receiving a vertical synchronizing signal, a horizontal synchro- 
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nizing signal and a clock signal of a fixed frequency for outputting a vertical transfer pulse to cause vertical transfer 
register 83 to execute charge transfer: and a horizontal driving circuit 82 receiving similar signals as vertical driving cir- 
cuit 81 for outputting a horizontal transfer pulse for driving charge transfer by horizontal transfer register 84. In synchro- 
nization with the vertical synchronizing signal, an output corresponding to the light intensity received at photoreceptor 
portion 85 is read to vertical transfer register 83. and in the period of the horizontal synchronizing signal, charges are 
transferred vertically row by row in vertical transfer register 83. In accordance with the clock signal period, charges are 
transferred column by column in horizontal direction in horizontal transfer register 84. 

Such driving operation of the CCD1 as described above is a well known operation for a so-called interline type 
CCD1 . The vertical and horizontal synchronizing signals as well as the clock signal are output from a timing pulse gen- 
erating circuit 71 shown in Fig. 8. 

Again referring to Fig. 8, the image signal output from CCD1 is subjected to known noise removal operation in a 
correlated double sampling circuit (hereinafter referred to as CDS circuit) 2. amplified by an auto gain control circuit 
(hereinafter referred to as AGC circuit) 3. and converted to a digital signal in an A/D converter 4. 

The digital image signal is applied as a first input signal directly to a color separation circuit 8, which is a two-dimen- 
sional non-recursive digital filter, as well as to a scan line delay provider (hereinafter referred to as 1H delay provider) 
5. An output from 1H delay provider 5 is input as a second input signal to color separation circuit 8 as well as to a 1H 
delay provider 6 of a succeeding stage. Further, an output from 1H delay provider 6 is input as a third input signal to 
color separation circuit 8 as well as to a 1 H delay provider 7 of a succeeding stage, and an output from 1 H delay pro- 
vider 7 is input, as a fourth input signal, to color separation circuit 8. 

Therefore, the first to fourth four input signals correspond to image signals of four scanning lines (four lines), and 
the signals from the four lines are collectively input to color separation circuit 8. 

Thus, an FIR (Finite Impulse Response) filter is implemented by color separation circuit 8 and three 1 H delay pro- 
viders 5. 6 and 7. 

Color separation circuit 8 includes ten 1 clockdelay providers 10, 11. 12, 13. 14. 15, 16. 17, 18, and 19 for providing 
a delay of 1 clock for the input signal: four multipliers 20. 21 , 22. and 23 for multiplying the value of an input signal by 2; 
fourteen selectors 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36 and 37 for selecting one of two input signals: and 
five adders 38. 39, 40. 41 and 42 for adding two input signals. 

The structure will be more specifically described. The first input signal from A/D converter 4 is input to 1 clock delay 
provider 1 0. and the output from delay provider 1 0 is directly input to terminal 24b of selector 24 as well as 1 clock delay 
provider 14. The output from 1 clock delay provider 1 4 is input to a terminal 24a of selector 24. 

The second input signal which is an output signal from 1H delay provider 5 is directly input to a terminal 25b of 
selector 25. as well as to 1 clock delay provider 1 1 . The output from 1 block delay provider 1 1 is input to 1 clock delay 
provider 1 5 of the succeeding stage, as well as to multiplier 20. The output from 1 clock delay provider 1 5 is input to 1 
clock delay provider 18 of the succeeding stage, as well as to multiplier 21 . The output from 1 clock delay provider 1 8 
is input to terminal 25a of selector 25. 

The third input signal, which is the output from 1 H delay provider 6 is directly input to terminal 26b of selector 26, 
as well as to l dock delay provider 12. The output from 1 clock delay provider 12 is input to 1 clock delay provider 16 
of the succeeding stage as well as to multiplier 22. The output from 1 clock delay provider 16 is input to 1 clock delay 
provider 19 of the succeeding stage, as well as to multiplier 23. The output from 1 clockdelay provider 19 is input to a 
terminal 26a of selector 26. 

The fourth input signal, which is the output from 1 H delay provider 7 is input to 1 clock delay provider 13. The output 
from delay provider 13 is directly input to a terminal 27b of selector 27 as well as to 1 dock delay provider 1 7. The output 
from 1 clock delay provider 1 7 is input to a terminal 27a of selector 27. 

The output from selector 24 is input to terminals 32a and 34a of selectors 32 and 34. respectively, of the succeed- 
ing stage. The output from selector 25 is input to terminals 32b and 34b of selectors 32 and 34. respectively The output 
from selector 26 is input to terminals 33a and 35a of selectors 33 and 35 of the succeeding stage. The output from 
selector 27 is input to terminals 33b and 35b of selectors 33 and 35, respectively 

The output from multiplier 20 is input to terminals 28a and 30a of selectors 28 and 30. respectively, of the succeed- 
ing stage. The output from multiplier 21 is input to terminals 28b and 31a of selectors 28 and 31. respectively of the 
succeeding stage. The output from multiplier 22 is input to terminals 29a and 30b of selectors 29 and 30, respectively, 
of the succeeding stage. The output from multiplier 23 is input to terminals 29b and 31b of selectors 29 and 31 . respec- 
tively, of the succeeding stage. 

The outputs from selectors 32 and 33 are added in adder 38 of the succeeding stage, and outputs from selectors 
34 and 35 are added in adder 39 of the succeeding stage. The output from selector 28 is input to terminals 36a and 37a 
of selectors 356 and 37. respectively of the succeeding stage. The output from selector 29 is input to terminals 36b and 
37b of selectors 36 and 37. respectively, of the succeeding stage. The outputs from selectors 30 and 31 are added in 
adder 40. 

The output from adder 38 is input, together with the output from selector 36, to adder 41, and added therein. The 
output from adder 39 is input, together with the output from selector 37, to adder 42. and added therein. 
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The output from adder 41 woufd finally be the R signal which has gone through color separation processing, the 
output from adder 42 would be the B signal and the output from adder 40 would be the G signal. 

In the above described flow of signal processing, it becomes possible to process signals of continuous four pixels 
of a corresponding line, as three 1 clock delay providers are arranged in series for each input signal. Signals of a 4 x 4 
pixel block can be handled if three 1 block delay providers are connected in series for each input signal. Switching of 
fourteen selectors Is controlled in the following manner by a switch control signal from switch control circuit 72. 

Assume that a 4 x 4 pixel block of CCD 1 includes 16 pixels represented by P 11 to P44 as shown in Fig. 9. for exam- 
ple. Color separation circuit 8 generates R. G and B signals at the centra! portion M denoted by a hatched circle of the 
sixteenth pixels shown in Fig. 9 by interpolation of R. G and B signals from sixteen neighboring pixels. 

More specifically, color separation circuit 8 generates R. G and B signals at a position obtained by shifting central 
four pixels of the 4 x 4 pixel block by half a pixel in horizontal and vertical directions, in other words, the position obtained 
by offsetting central four pixels by half a pixel in horizontal and vertical directions, by using R. G and B signals from 
some of the neighboring sixteen pixels. 

Consider sixteen pixels shown in Fig. 9. First, signals from lowermost one line of pixels are successively output 
from horizontal transfer register 84 of CCD1 at every 1 clock, namely. P1 1 — > P12 — > P13 P14. When output of the 
signals of all the pixels in this line is completed, signals from pixels of the second lowest line are output namely P21 - 
^ P22 P23 — > P24. 

Thereafter, signals from the third from the bottom line are output, namely, P31 — > P32 — > P33 > P34. 

When the output of signals from the pixels of the third line from the bottom is completed, signals from respective 
pixels in the uppermost line are successively output, namely. P41 P42 — > P43 — ^ P44. 

For convenience of description, in the following, the signal obtained by photoelectric conversion from pixel Pi 1 will 
be denoted by Si 1 , the signal from pixel P12 will be denoted by Si 2 and similarly, signals output from pixels up to P44 
will be denoted by the reference characters up to S44. 

The image signals Si 1 to S44 are passed through CDS circuit 2 and AGC circuit 3 and successively converted to 
digital values by A/D converter 4. 

By the time the output of four lines is completed and the signal S44 from pixel P44 is output from A/D converter 4. 
one line of signals, that is image signals S1 1 to Si 4 are input to color separation circuit 8 from 1 H delay provider 7. and 
one line of signals, that is, image signals S21 to S24 are input to color separation circuit 8 from 1H delay provider 6. 
Similarly, one line of signals, that is. image signals S31 to S34 are input to color separation circuit 8 from 1 H delay pro- 
vider 5, and one line of signals, that is, image signals Si 1 to S1 4 are directly input to color separation circuit 8. 

Therefore, signal S12 is output from 1 clock delay provider 17, signal S13 is output from 1 clock delay provider 13. 
and signal S 1 4 is output from 1 H delay provider. Similarly, signals S2 1 . S22 and S23 are output from 1 clock delay pro- 
viders 19. 16 and 12. respectively, and signal 824 is output from 1H delay provider 6. Signals 831. S32 and 833 are 
output from 1 clock delay providers 1 8. 1 5 and 11 . respectively, and signal 834 is output from 1 H delay provider 5. Sig- 
nals 842 and 843 are output from 1 clock delay providers 14 and 16. respectively, and signal S44 is output from A/D 
converter 4. 

In the structure of color separation circuit 8 described above, in order to perform interpolation in color separation 
circuit 8. a pixel block consisting of 16 pixels, that is, 4 x 4. on CCD1 must be set. Fig. 14 shows positions where inter- 
polation takes place with respect to the arrangement of the mosaic color filter shown in Fig. 2. As already described. 4 
x 4. that is. 16 pixels are necessary for interpolation. Therefore, as shown in Fig. 14. the result of interpolation when the 
image signals from pixels of lowermost three rows are output is meaningless. Further, interpolation at the time of output 
of first three pixels of the fourth row from the bottom is similarly meaningless. More specifically, the result of interpola- 
tion at interpolation portion Y comes to have meaning for the first time when the image signal of pixel X is obtained in 
Fig. 14. » a K . 

Therefore, switching control of respective selectors for interpolation must be started from the time point when the 
image signal from pixel X is output from CCD1 . 

Here, there are four possible arrangements as shown in Figs. 10 to 13 of 4x4=16 pixels for the microcolor filters 
shown in Fig. 14. 

Figs. 10A to 13A show arrangements of color filter arrays of 4 x 4 = 16 pixels. Figs. 10B to 13B show weight coef- 
ficients for respective pixels when the G signal at the central point is to be generated by interpolation. Figs. IOC to 13C 
show weight coefficients for respective pixels when the R signal at the central position is to be generated by interpola- 
tion. Figs. 10D to 13D show weight coefficients for respective pixels when the B signal at the central position is to be 
generated by interpolation. 

Referring to Fig. 14. in the process of interpolation at interpolation portion Y performed as a G signal is read from 
pixel X. 16 pixels surrounding interpolation portion Y at the center would be as shown in Fig. 10A. Therefore, for the 
interpolation of interpolation portion Y. switching control appropriate for arrangement 1 of respective selector' circuits 
becomes necessary. 

When B signal is read from a pixel next to the pixel X in the right, interpolation at an interpolation portion adjacent 
to interpolation portion Y on the right becomes possible. The arrangement of sixteen pixels surrounding this interpola- 
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tion portion corresponds to arrangement 2 and hence switch control appropriate for arrangement 2 of respective selec- 
tor circuits becomes necessary. 

Thereafter, as long as image signals are output from the pixels of this line, arrangement of sixteen pixels surround- 
ing each interpolation portion would be arrangements 1 and 2 switched alternately. Therefore, relative to this switching, 
selectors must be controlled to be switched to the state suitable for the respective arrangements. 

As for the next line, that is, a line upper than the line to which pixel S belongs, at a time point when R signal is read 
from the fourth from the left pixel, interpolation at an interpolation portion adjacent to interpolation portion Y on the 
upper side becomes possible. The arrangement of sixteen pixels surrounding this interpolation portion corresponds to 
arrangement 3 shown in Fig. 12A. Therefore, selector circuits must be switched to be suitable for an-angement 3. Fur- 
ther, when reading of the G signal from the adjacent right pixel is completed, interpolation at interpolation portion adja- 
cent to interpolation portion Y on upper right position becomes possible. The arrangement of sixteen pixels surrounding 
this interpolation portion corresponds to arrangement 4 shown in Fig. 1 3A. Therefore, selector circuits must be switched 
to be suitable for arrangement 4. 

Thereafter, while the image signals are output from the pixels of this line, the arrangement of sixteen pixels sur- 
rounding the corresponding interpolation portions correspond arrangements 3 and 4 switched alternately. Therefore, 
selector circuits must be controlled to be switched to suitable states for respective arrangements, correspondingly 

In the next row. after the time point at which the image signal from the fourth pixel is input, switching control suitable 
for arrangements 1 and 2 is executed. Similarly in the next row, after the time point at which the image signal from the 
fourth pixel is input, switching control suitable for arrangements 3 and 4 is executed. 

Thereafter, control for switching arrangements 1 and 2 alternately and arrangements 3 and 4 alternately for every 
other row is continued until reading of image signals from the pixels of the uppermost row is completed. 

When reading of image signals of one image plane is completed, stored charges at the photoreceptor portion are 
again read to the vertical register and reading from the lowermost line of pixels is again performed, similar switching 
control as described above is repeated. 

In order to control switching of respective selectors as described above, actually it is necessary to determine the 
position of the pixel on the CCD1 from which image signal is being ou^ut. by means of a vertical counter and a hori- 
zontal counter included in switch control circuit 72. 

The vertical counter is reset by the vertical synchronizing signal, counts the horizontal synchronizing signal and 
hence counts the line to which the pixel being processed belongs. The horizontal counter is reset by the horizontal syn- 
chronizing signal, counts the clock signals synchronized with horizontal transfer, and determines to which column in the 
horizontal direction the pixel being processed belongs. By these two counters, that is, vertical and horizontal counters, 
the position of the pixel from which image signal is being output on the CCD1 is determined. For example, if it is deter- 
mined that the image signal from a pixel at the fourth row from the bottom and fourth column from the left is being output 
from CCD1, switch control circuit 7 determines that the arrangement of 4 x 4 = 16 pixels used for interpolation corre- 
sponds to arrangement 1 . and outputs a switch control signal corresponding to arrangement 1 . As long as the output of 
pixels belonging to this line continues, switch control signals corresponding to arrangements 1 and 2 are output alter- 
nately in the period of clock signal. 

If it is determined by the vertical and horizontal counters that the position on the CCD1 of that pixel from which 
image signal is being output is the pixel at the fifth row from the bottom and fourth column from the left, switch control 
circuit 72 determines that the arrangement of 1 6 pixels used for interpolation corresponds to arrangement 3 and outputs 
a switch control signal corresponding to arrangement 3. As long as the output of image signals from pixels belong to 
this line continues, switch control signals corresponding to arrangements 3 and 4 are output afternately in the period of 
the clock signal. 

The image signal output from the corresponding pixel on the CCDl passes through CDS circuit 2, AGC circuit 3 
and A/D converter 4. before it is input to color separation circuit 8. Therefore, after the delay of time necessary for these 
processings, the switch control signal is output from switch control circuit 72 to respective selector circuits. The clock 
signal is in synchronization with charge transfer by horizontal transfer register 84 of CCDl, and it also serves as a driv- 
ing clock of color separation circuit 8. 

Fig. 16 shows correspondence between switch control signals from switch control circuit 72 and the arrangement 
of sixteen pixels. 

In accordance with the corresponding pixel arrangement of the image signals input to color separation circuit 8. 
selector circuits 24 to 37 are switched as shown in Fig. 16. and interpolated R. B and G signals are output from color 
separation circuit 8. 

The color separation operation in the color separation circuit 8 described above is as follows. 

First, the procedure for generating G signal will be described. When a G signal is to be generated at interpolation 
portion Y which is the central portion of 4 x 4 = 1 6 pixels at which interpolation takes place. G signals obtained from two 
of four pixels surrounding interpolation portion Y are used. 
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More specifically, in the arrangement 1 shown in Fig. 10A and in the arrangement 4 shown in Fig. 13A. G signals 
from pixels P22 and P33 are multiplied by the weight coefficient of "2" as shown in Figs. 1 0B and 13B, and the multiplied 
values are added, whereby the G signal at the interpolation portion Y is generated. 

In the arrangement 2 shown in Fig. 11 A and in the arrangement 3 shown in Fig. 12A.'G signals from pixels P32 and 
P23 are allotted with weight coefficient of "2" as shown in Rgs. 118 and 12B, and similar calculation is performed to 
generate the G signal for the interpolation portion Y. 

In the above described processing, interpolation is performed by a two-dimensional two-tap filter including 1 clock 
delay providers 1 1 . 12. 15. 16. multipliers 20. 21. 22. 23. selector circuits 30. 31 and adder 40 in color separation circuit 
8. More specifically, for the arrangements 1 and 4. selectors 30 is switched to the side of terminal 30a by the switching 
signal, so that the output from multiplier 20 is selected, and selector 31 is switched to that terminal 31b to select the 
output from multiplier 23. The signal S33 which is the output from 1 clock delay provider 1 1 is multiplied by 2 in multiplier 
20, and input to adder 40 by selector 30. f^eanwhile. the signal S22 which is the output signal from 1 clock delay pro- 
vider 16 is multiplied by 2 in multiplier 23. and input by selector 31 to adder 40. In adder 40. these inputs are added, 
and thus G signal at interpolation portion Y is generated. 

Meanwhile, for the arrangements 2 and 4. switching is controlled such that selector 30 is switched to terminal 30b 
to select the output from multiplier 22. and selector 31 is switched to terminal 31a to select the output from multiplier 21 . 
The signal S23 which is the output from 1 clock delay provider 12 is multiplied by 2 in multiplier 22 and input by selector 
30. to adder 40. The signal S32 which is the output signal from 1 clock delay provider 15 is multiplied by 2 in multiplier 
21 and input by selector 31 . to adder 40. In adder 40. these inputs are added, whereby the G signal at the interpolation 
portion Y is generated. 

More specifically, in interpolation processing of G signal, of 4 x 4 pixels, only the image signals from central 2x2 
pixels are used to generate the G signal. 

The procedure for generating R signal will be described. 

When the R signal is to be generated at interpolation portion Y, one of the four pixels suri-ounding interpolation por- 
tion Y and two pixels of R color filters out of the outermost 12 pixels belonging to the same row or same column as the 
said pixel are used, that is. R signals from the total of three pixels are used. 

More specifically, for the arrangement 1 . the weight coefficient of "2*' is allotted to the R signal from pixel P32 as 
shown in Fig. IOC. and weight coefficient of "1" is allotted to the R signals from pixels P 12 and P34. R signals from 
these three pixels are multiplied by respective weight coefficients, and the multiplied values are added to provide the R 
signal of the central portion Y. 

Similarly, for the arrangement 2. the weight coefficient of "2" is allotted to R signal from pixel P33 as shown in Fig. 
lie, and weight coefficient of "ris allotted to the R signals from pixels PI 3 and P31. The R signals from these three 
pixels are multiplied by respective weight coefficients, and the multiplied values are added to provide the R signal of the 
central portion Y 

For the arrangement 3, weight coefficient of "2" is allotted to R signal from pixel P22 and weight coefficient of "1 " is 
allotted to the R signals from pixels P24 and P22. as shown in Fig. 1 2C. The R signals from these three pixels are mul- 
tiplied by respective weight coefficients, and the multiplied values are added to provide the R signal at the central por- 
tion Y 

For the arrangement 4. weight coefficient of "2" is allotted to the R signal of pixel P23, and weight coefficient of "1 " 
is allotted to the R signals from pixels P21 and P43. as shown in Fig. 13C. The R signals from these three pixels are 
multiplied by respective weight coefficients, and the multiplied values are added to provide the R signal at the central 
portion Y. 

In this manner, when R signal is to be calculated by weighting and adding, a two-dimensional three-tap filter includ- 
ing all the 1 clock delay providers, multipliers 20, 21 . 22 and 23, selector circuits 24. 25. 26, 27, 28. 29. 32. 33 and 36 
and adders 38 and 41 of color separation circuit 8 is used. 

More specifically, for the arrangement 1 , selectors 27 and 36 are switched to the a terminals, respectively, and 
selectors 25. 28. 32 and 33 are switched to b terminals, respectively. Therefore, the signal value S32 is multiplied by 2 
in multiplier 21, and input through selectors 28 and 36 to adder 41. The signal value S1 2 is input through selectors 27 
and 33 to adder 38. The signal value S34 is input through selectors 25 and 32 to adder 38. Therefore, from adder 38. 
a signal (S12 + S34) is output. Finally, from adder 41 . a signal (2 x S32 + S12 + S34) would be output, which is the R 
signal at the interpolation portion Y 

For the arrangement 2. selectors 25, 28. and 38 are switched to a terminals, respectively, and selectors 25, 32 and 
33 are switched to b terminals, respectively Consequently, the signal value S33 is multiplied by 2 in multiplier 20. and 
input to adder 41 through selectors 28 and 36. The signal value S13 is input through selectors 27 and 33 to adder 38. 
Signal value S31 is input to adder 38 through selectors 25 and 32. Therefore, a signal (S13 + S31) is output from adder 
38. Finally, a signal (2 x S33 + S13 + S31) would be output from adder 41, which is the R signal at interpolation portion 

For the arrangement 3, selectors 24, 32 and 33 are switched to a terminals, respectively, and selectors 26, 29 and 
36 are switched to b terminals, respectively Consequently, the signal value S22 is multiplied by 2 in multiplier 23 and 
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input through selectors 29 and 36 to adder 41 . The signal value S24 is input through selectors 26 and 33 to adder 38. 
The signal value S42 is input through selectors 24 and 32 to adder 38. Therefore, a signal (S24 + S42) is output from 
adder 38. Finally, a signal (2 x S22 + S24 + S42) would be output from adder 41. which is the R signal at the interpola- 
tion portion Y 

For the arrangement 4. selectors 26. 29, 32, and 33 are switched to a terminals, respectively, and selectors 24 and 
36 are switched to b ternninals. respectively. Consequently, the signal value S23 is multiplied by 2 in multiplier 22, and 
input through selectors 29 and 36 to adder 41. The signal value S21 is input to adder 38 through selectors 26 and 33. 
The signal value S43 is input through selectors 24 and 32 to adder 38. Therefore, a signal (S21 + S43) is output from 
adder 38. Finally, a signal (2 x S23 + S21 + S43) would be output from adder 41. which will be the R signal at interpo- 
lation portion Y. 

The procedure for generating the B signal will be described in the following. 

For generating the B signal at interpolation portion Y. one of the four pixels surrounding the interpolation portion Y 
and two pixels out of outermost 12 pixels belonging to the same row or same column as the said pixel, are used, that 
is. B signals from the total of these three pixels are used. More specifically, for the arrangement 1 , weight coefficient of 
"2" is allotted to the B signal from pixel P23 and weight coefficient of "1" is allotted to the B signals from pixels P21 and 
P43 as shown in Fig. 10D. The B signals from these three pixels are multiplied by respective weight coefficients and the 
multiplied values are added, whereby the B signal of the interpolating portion Y is calculated. 

Similarly, for the arrangement 2, weight coefficient of "2" is allotted to the B signal from pixel P22 and weight coef- 
ficient of "1" Is allotted to the B signals from pixels P24 and P42, as shown in Fig. 1 1 D. The B signals from these three 
pixels are multiplied by respective weight coefficients and the multiplied values are added, whereby the B signal at the 
interpolation portion Y is calculated. 

For the arrangement 3. weight coefficient of "2" is allotted to the B signal from pixel P33 and weight coefficient of 
"1" is allotted to the B signals from pixels P13 and P31 as shown in Fig. 12D. The B signals from these three pixels are 
multiplied by respective weight coefficients, and the multiplied values are added, whereby the B signal at the interpolat- 
ing portion Y is calculated. 

For the arrangement 4. weight coefficient of "2" is allotted to the B signal from pixel P32 and weight coefficient of 
"r is allotted to B signals from pixels P12 and P34. as shown in Fig. 13D. The B signals from these three pixels are 
multiplied by respective weight coefficients, and by adding these multiplied values, the B signal of the interpolating por- 
tion Y is calculated. 

In this manner, when the B signal is to be calculated by weighting and adding, a two-dimensional three-tap filter 
including all the 1 clock delay providers, multipliers 20. 21 , 22 and 23, selector circuits 24. 25, 26, 27, 28, 29, 34, 35, 37 
and adders 39 and 42 of color separation circuit 8 is used. 

More specifically, for the arrangement 1 . selector circuits 26, 29, 34 and 35 are switched to a terminals, respectively, 
and selector circuits 24 and 37 are switched to b terminals, respectively. Therefore, the signal value S23 is multiplied by 
2 in multiplier 22. and input through selectors 29 and 37 to adder 42. The signal value S21 is input through selectors 26 
and 35 to adder 39. The signal value S43 is input through selectors 24 and 34 to adder 39. Therefore, a signal (S21 + 
S43) is output from adder 39. Finally, a signal {2 x S23 + S21 + S43) would be output from adder 42, which will be the 
B signal at interpolation portion Y 

For arrangement 2. selectors 24, 34, and 35 are switched to a terminals, respectively, and selector circuits 26, 29 
and 37 are switched to b terminals, respectively Consequently, the signal value S22 is multiplied by 2 in multiplier 23 
and input to adder 42 through selector circuits 29 and 37. The signal value S24 is input through selector circuits 26 and 
35 to adder 39. The signal value S42 is input through selector circuits 24 and 34 to adder 39. Therefore, a signal (S24 
+ S42) is output from adder 39. Finally, a signal (2 x S22 + S24 + S42) would be output from adder 42 which is the B 
signal at interpolation portion Y 

For the arrangement 3, selector circuits 25. 28 and 37 are switched to a terminals, respectively, and selecting cir- 
cuits 27. 34 and 35 are switched to b terminals, respectively. Therefore, the signal value S33 is multiplied by 2 in multi- 
plier 20 and input to adder 42 through selectors 28 and 37. The signal value S13 is input through selector circuits 27 
and 35 to adder 39. The signal value S31 is input through selector circuits 25 and 34 to adder 39. Therefore, a signal 
(S13 + S31) is output from adder 39. Finally, a signal (2 x S33 -i- S13 + S31) would be output from adder 42. which is 
the B signal at interpolation portion Y. 

For the arrangement 4. selector circuits 27 and 37 are switched to a terminals, respectively, and selector circuits 
25. 28, 34 and 35 are switched to b terminals, respectively. Consequently, the signal value S32 is multiplied by 2 in mul- 
tiplier 21 . and input to adder 42 through selector circuits 28 and 37. The signal value S34 is input through selector cir- 
cuits 25 and 34 to adder 39. Therefore, a signal (Si 2 + S34) is output from adder 39. Finally, from adder 42, a signal (2 
X 832 + S12 + 834) would be output, which is the B signal at interpolating portion Y 

In the interpolation of R, G and B signals described above, it is also possible to utilize weighted mean in which color 
signal values for each arrangement is divided by 4. i.e.. the sum of weight coefficients. 
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In this manner, regardless of which of the four different arrangements corresponds to the arrangement of an arbi- 
trary 4x4 pixel block on CCDl . three primary signals of R. G and B can be calculated at interpolation portion Y of the 
pixel block in color separation circuit 8. 

When generation of color signals at interpolation portion Y of a certain 4x4 pixel block is completed and image 
signal from the next pixel is output from A/D converter 4, the object pixel block is shifted by one pixel in horizontal direc- 
tion, and similar processing is repeated. When this horizontal movement results in complete movement of one row in 
CCD 1 . the pixel block of which interpolation is to be performed returns to the initial position horizontally while it is shifted 
by one pixel in vertical direction. 

As the pixel block is shifted, the interpolation portion Y is also shifted in horizontal and vertical directions succes- 
Sfvely. Finally, as shown in Fig. 14. R. G and B signals at an intersection of four pixels on the CCD1 are calculated. Here, 
the interpolatron portion marked by the hatching in Fig. 1 4 indicates a central point of any pixel block if a 4 x 4 pixel block 
can be actually formed on the CCDl . 

Other than the hatched portion, it is possible to calculate values corresponding to R. G and B signals, based on the 
signals output from CCD1 successively from color separation circuit 8. However, a 4 x 4 pixel block cannot be physically 
set on the CCDI, and hence the calculated value is meaningless as data for interpolation. 

Generally, on CCDI . non-effective pixels which are not displayed on a monitor are arranged on left, right, upper and 
lower edges. Therefore, if the interpolation portions denoted by the hatched portions in Fig. 14 are used as effective 
pixels and portions not hatched are set as non-effective pixels, only the color signals obtained from the effective pixels 
would be visible signals. 

In Fig. 14. the pixels of CCD1 have been described as having a scale of 8 x 8 for convenience of description. There- 
fore, hatched portions are smaller as compared with portions not hatched. However, generally, the total number of pix- 
els of a CCDI is as large as about 530 x 500. and most of the total number of pixels can be occupied by the hatched 
portions, that is. portions where 4x4 pixel blocks can be set. In other words, most pixels can be used as effective pixels. 

The transfer function of a two-dimension non-recursive digital filter for performing interpolation of color signals in 
color separation circuit 8 in the above described manner are as follows. 

[ARRANGEMENT 1] 



(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H (z) = £hmz ■"'=1xz-%1xz-"'=l+z''' 
(HORIZONTAL DIRECTION OF R SIGNAL) 

H (z) = Lhmz'"'=0xz'°+3xz'''+0x0'^+1x0'^=3z'Vz' 
(VERTICAL DIRECTION OF R SIGNAL) 

H (z) = Lhmz -"^=1 xz ■°+Oxz +3xz '^Oxz '^=1 +3z 
(HORIZONTAL DIRECTION OF B SIGNAL 



H (z) = !:hmz-'^=1xz-°+0xz-V3xz-^+0xz -^ = 1+3z'^ 
(VERTICAL DIRECTION OF B SIGNAL) 

H (z) = Ihmz '^=0x2 *%3xz +Oxz '^l xz '^=3z '\z 

(ARRANGEMENT 2] 



(HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H (z) = Ehmz 1 xz xz = 1 +z 
(HORIZONTAL DIRECTION OF R SIGNAL) 

H (z) = Lhmz-'"=1xz-°+0xz-''^xz'^0xz-='.=1+3Z-2 
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(VERTICAL DIRECTION OF R SIGNAL) 

H (z) = Lhmz -"^=1x2 *°+Oxz ^xz '^Oxz "^=1+32 ' 
5 (HORIZONTAL DIRECTION OF B SIGNAL) 

H (z) ^ Lhmz -"^=0x2 -\3x2 Voxz '^l xz "^Sz 'Vz 
(VERTICAL DIRECTION OF B SIGNAL) 

70 

H (2) = Ehmz -^=0x2 -°+3xz "Voxz "^l xz ^3z +z 

[ARRANGEMENT 3) 
15 (HORIZONTAL & VERTICAL DIRECTIONS OF G SIGNAL) 

H (2) = i:hm2"'"=1x2*\lxz-''=1+2'' 

20 

H (2) = Lhm2 •'"=0x2 ■°+3x2 ^0x2 '^l xz '^32 ^2 
(VERTICAL DIRECTION OF R SIGNAL) 

H (z) = Lhmz -"'=0x2 -^3x2 +0x2 '^1 X2 -=^=32 +2 
(HORIZONTAL DIRECTION OF B SIGNAL) 

H (2) = Lhms -"^=1 X2 ■°+0x2 +3X2 -^+0x2 "^1+32 -2 

30 

(VERTICAL DIRECTION OF B SIGNAL) 

H (2) Lhms-^=1x2*°+0xz •''+3X2 -2+0x2-^=1+32 -2 

35 [ARRANGEMENT 4] 

(HORIZONTAL & VERTICAL SIGNALS OF G SIGNAL) 

H (2) = Lhm2 -^"=1x2 -*'+1 X2 =1 +2 

40 

(HORIZONTAL DIRECTION OF R SIGNAL) 

H (2) = Lhmz ■'"=1 x2 ■°+0x2-''+3x2 -^+0x2 •^'1+32 
45 VERTICAL DIRECTION OF R SIGNAL) 

H (2) = lhmz '"^=0x2 -^Sxz +Oxz -^1 xz •"=32 *Vz 
(HORIZONTAL DIRECTION OF B SIGNAL) 

50 

H (z) = Lhmz •"^=0x2-0+3x2 +0x2 -^+1 X2 *^=32 +2 
(VERTICAL DIRECTION OF B SIGNAL) 

H (2) = Lhm2-m=1x2'^+0x2'V3x2 •2+0x2 •=^=1+32^2 

"The frequency characteristics of color signals which are the output signals of the two^Jimensional non-recursive 
drgital filter having such transfer functions are as shown in Fig. 15. In the graph, the curve Ql shows hori2ontal and ver- 
tical Characteristics of G signal, and curve Q2 shows hori2ontal and vertical characteristics of R and B signals 
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As is apparent from Figs. 10B to 14B. the pixels used for generating the G signal exist at the central two columns 
out of four columns when viewed horizontally, and exist in central two columns when viewed vertically. Therefore, in any 
of the arrangements 1 to 4, the gain lowers near the Nyquist frequency. 

The sum of the weight coefficients of the pixels used for generating the R signal, when viewed horizontal, any of 
the central two columns out of four columns is "3". and in the column outer by one column, it would be "1". Meanwhile, 
when viewed vertically, either of the central two rows out of four rows is always "3". and an outer row spaced by one rovJ 
is "I ". Therefore, frequency characteristic of R signal is always as represented by Q2. 

Similarly to R signals, the sum of the weight coefficients of the pixels used for generating the B signal is, when 
viewed horizontally, either of the central two of four columns is always "3" and an outer column spaced by one column 
is -r. When viewed vertically, it is always "3" in either of the central two rows out of four rows, and in the outer row 
spaced by one row. it is "r. Therefore, the frequency characteristic of the B signal is always as represented by Q2. 

As is apparent from Fig. ^ 5, the frequency characteristic Q 1 of the G signal which has highest contribution to bright- 
ness among three primary signals suffers from smaller attenuation in the higher frequency range as compared with the 
curve P2 shown in Fig. 8, which means that higher resolution is possible. 

Further, at 1/2 Nyquist frequency, the difference between frequency characteristic of R and B signals over the fre- 
quency characteristic of G signal is considerably smaller than in Fig. 8, and hence generation of a color false signal can 
be suppressed. 

Therefore, according to the present invention, a frequency characteristic which suffers from smaller attenuation in 
high frequency range can be provided for the G signal which has highest contribution to brightness, so that high reso- 
lution can be obtained. Further, at 1/2 Nyquist frequency, the difference between frequency characteristic of R and B 
signals and G signal is small, and hence generation of a color false signal can be suppressed. Further, the same fre- 
quency characteristic can be obtained for every pixel, for each of the three primaries. 

Although the present invention has been described and illustrated in detail, it is clearly understood that the same is 
by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the present 
invention being limited only by the terms of the appended claims. 

Claims 



1 . A 1 -chip color video camera. (1 00) comprising: 

solid-state image sensing means (1) provided with photoelectric converting elements corresponding to 
respective pixels arranged in an array (85). 

said solid-state image sensing means including on its light receiving surface a color filter array (85) in which 
color filters of primaries are placed in a prescribed arrangement corresponding to said photoelectric converting ele- 
ments: and 

interpolating means (5. 6. 7. 8) receiving an output from said solid-state image sensing means for outputting 
corresponding color signals. 

said interpolating means including 

parallel color signal output means (5, 6. 7) receiving said output from solid-state image sensing means for 
outputting color signals corresponding to a prescribed even number of rows of said pixels in synchronization with 
and parallel to each other successively, column by column, 

control means (72) for outputting a synchronized interpolation designating signal in accordance with corre- 
spondence between color signals output column by column from said parallel color signal output means and 
arrangement of predetermined weight coefficients for the prescribed arrangement of said color filter array, and 

color separation means (8) receiving said outputs from said parallel color signal output means for performing 
interpolation of color signals from a pixel block including pixels of said prescribed even number rows and said pre- 
scribed even number columns in accordance with said interpolation designating signal, and for successively out- 
putting color signals corresponding to a central position of said pixel block in synchronization. 

2. A 1-chip color video camera, comprising: 

solid-state image sensing means provided with photoelectric converting elements corresponding to respec- 
tive pixels arranged in an array. 

said solid-state image sensing means including on its light receiving surface a color filter array in which color 
filters of primaries are arranged mosaic-wise corresponding to said photoelectric converting elements. 

said color filter array having green filters placed in diagonal direction in an arbitrary color filter arrangement 
of two rows by two columns; and 

interpolating means receiving an output from said solid-state image sensing means for interpolating green 
signal component at a central portion of a pixel block corresponding to said arbitrary color filter arrangement of two 
rows by two columns, using mean value of green signals from those of said photoelectric converting elements 
which correspond to said green filters placed in the diagonal direction. 
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A 1-chip color video camera, comprising: 

solid-state image sensing means provided with photoelectric converting elements corresponding to respec- 
tive pixels arranged in an array, 

said solid-state image sensing means including on its light receiving surface a color filter array in which color 
filters of primaries are arranged mosaic-wise corresponding to said photoelectric converting elements. 

said color filter array having red filters and blue filters arranged checkerwise. rows including said red filters 
and rows including said blue filters being arranged alternately; and 

interpolating means for interpolating, based on signals from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of red and blue at a central portion of said pixel block. 

said Interpolating means including a two-dimensional non-recursive digital filter circuit (5. 6, 7, 8) for switch- 
ing, in accordance with arrangement of color filters corresponding to said sixteen pixels, values of sums of weight 
coefficients of every column in vertical direction to [0, 3. 0. 1] or [1 . 0. 3, 0]. 

A 1-chip color video cameral, comprising: 

solid-state image sensing means provided with photoelectric converting elements corresponding to respec- 
tive pixels arranged in an array. 

said solid-state image sensing means including on its light receiving surface a color filter array in which color 
filters of primaries are arranged mosaic -wise corresponding to said photoelectric converting elements. 

said color filter array having red filters and blue filters arranged checkenwise. rows including said red filters 
and rows including said blue filters being arranged alternately; and 

interpolating means for interpolating, based on signals from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of red and blue at a central portion of said pixel block. 

said interpolating means including a two-dimensional non-recursive digital filter for switching, in accordance 
with arrangement of the color filters corresponding to said sixteen pixels, values of sums of weight coefficients of 
every row in horizontal direction to [0. 3, 0, 1] or [1, 0, 3, 0]. 

A 1-chip color video camera, comprising: 

solid-state image sensing means provided with photoelectric converting elements corresponding to respec- 
tive pixels arranged in an array. 

said solid-state image sensing means including on its light receiving surface a color filter array in which color 
filters of primaries are arranged mosaic-wise corresponding to said photoelectric converting elements 

said color filter array having red filters and blue filters arranged checkenrt^ise, rows including said red filters 
and rows including said blue filters being arranged alternately; and 

interpolating means for interpolating, based on signals from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of red and blue at a central portion of said pixel block, 

said interpolating means including a two-dimensional non-recursive digital filter circuit. 

wherein 

when said pixel block of sixteen pixels includes sixteen pixels of four rows by four columns from (m. n) to (m 
-»■ 3, n + 3) with m and n being integers. 

said two-dimensional non-recursive digital filter circuit performs interpolation by 

setting weight coefficient of a pixel (m + 1 , n + 1) to 2 and weight coefficient of each of pixels (m + 1 , n 3) 
and (m + 3, n + 1 ) to 1 , when either red or blue color filter is placed on the pixel (m + 1 . n + 1), 

setting weight coefficient of a pixel (m + i . n + 2) to 2 and weight coefficient of each of pixels (m + 1 . n) and 
(m + 3, n + 2) to 1 . when either red or blue color filter is placed on the pixel (m + 1. n + 2), 

setting weight coefficient of a pixel (m + 2. n + 1) to 2 and weight coefficient of each of pixels (m, n + 1 ) and 
(m + 2. n + 3) to 1 , when either red or blue color filter is placed on the pixel (m + 2. n + 1), and 

setting weight coefficient of a pixel (m + 2. n + 2) to 2 and weight coefficient of each of pixels (m. n + 2) and 
(m + 2. n) to 1 . when either red or blue color filter is placed on the pixel (m + 2, n + 2). 

The 1 -chip color video camera according to claim 1, further comprising: 

analog-digital converting means (4) receiving said output from said solid-state image sensing means for out- 
putting it as a corresponding digital signal; wherein 

said parallel color signal output means includes (2k - 1) line memories (5. 6, 7), where 2k represents said 
prescribed even number. 

each said line memory having transfer capacity corresponding to a row of said pixels, and said line memo- 
ries being connected in series to receive as an initial stage input, an output from said analog-digital converting 
means, and 

said parallel color signal output means outputs said output from said analog-digital converting means and 
outputs from said line memories parallel to each other; and 
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said color separation means includes 

a two-dimensional transfer register (10-19) receiving said parallel color signal outputs for transferring them 
to a prescribed direction successively, and holding at most 2k x 2k color signal values, and 

interpolation calculation means (72, 20 - 42) responsive to said interpolation designating signal for adding 
corresponding ones of said color signals held in said two-dimensional transfer register to each other, in accordance 
with arrangement of said weight coefficients. 

The 1 -chip color video camera according to claim 6, wherein 

said color filter array includes green filters arranged in a diagonal direction in an arbitrary color filter arrange- 
ment of two rows by two columns; and 

said interpolation calculation means outputs, in accordance with said interpolation designating signal, a 
green signal corresponding to a central position of said pixel block as either a mean value of color signal values of 
(k, k) and (k + 1. k + 1) or a mean value of color signal values of (k + i, k) and (k. k + 1), of the two-dimensional 
arrangement of said 2k x 2k color signal values. 

The 1 -chip color video camera according to claim 6. wherein 
said prescribed even number is 4; 

said color filter array includes red filters and blue filters arranged checkenA/ise. rows including said red filters 
and rows including said blue filters being arranged alternately; 

said two-dimensional transfer register holds signals from a pixel block including sixteen pixels of four rows 
by four columns; and 

said interpolation calculation means switches, in accordance with said interpolation designating signal, val- 
ues of sums of weight coefficients in every column in vertical direction for color signals corresponding to said six- 
teen pixels to [0. 3, 0. 1] or [1 , 0. 3, 0], for interpolating and outputting color signal component of each of red and 
blue at a central portion of said pixel block. 

The 1-chip color video camera according to claim 6, wherein 
said prescribed even number is 4; 

said color filter array includes red filters and blue filters arranged checkenwise. rows including said red filters 
and rows including said blue filters being arranged alternately; 

said two-dimensional transfer register holds signals from a pixel block including sixteen pixels of four rows 
by four columns; and 

said interpolation calculation means switches, in accordance with said interpolation designating signal, val- 
ues of sums of weight coefficients of every row in horizontal direction for color signals corresponding to said sixteen 
pixels to [0. 3. 0. 1] or [1 , 0. 3. 0], for interpolating and outputting color signal component of each of red and blue at 
a central portion of said pixel block. 

The 1 -chip color video camera according to claim 6, wherein 
said prescribed even number is 4; 

said color filter array includes red filters and blue filters arranged checkenwise. rows including said red filters 
and rows including said blue filters being arranged alternately: 

said two-dimensional transfer register holds signals from a pixel block including sixteen pixels of four rows 
by four columns: and 

said interpolation calculation means performs interpolation in accordance with said interpolation designating 
signal, where said pixel block of 16 pixels includes sixteen pixels of four rows by four columns of (m n) to (m + 3 n 
+ 3). by 

setting weight coefficient of a pixel (m + 1 . n + 1) to 2 and weight coefficient of each of pixels (m + 1 . n + 3). 
and (m + 3, n + 1 ) to 1 . when either red or blue color filter is placed on the pixel (m -i- 1 , n + 1 ). 

setting weight coefficient of a pixel (m + 1 , n + 2) to 2 and weight coefficient of each of pixels (m + 1 , n) and 
(m + 3. n + 2) to 1 , when either red or blue color filter is placed on the pixel (m + 1 , n + 2). 

setting weight coefficient of a pixel (m + 2. n + 1) to 2 and weight coefficient of each of pixels (m, n + 1) and 
(n + 2. n + 3) to 1 . when either red or blue color filter is placed on the pixel (m + 2. n + 1), and 

setting weight coefficient of a pixel (m + 2. n + 2) to 2 and weight coefficient of each of pixels (m, n + 2) and 
(m + 2. n) to 1 . when either red or blue color filter is placed on the pixel (m + 2, n + 2). 

A 1 -chip color video camera, comprising: 

solid-state image sensing means provided with photoelectric converting elements corresponding to respec- 
tive pixels arranged in an array. 

said solid-state image sensing means including on its light receiving surface a color filter array in which color 
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filters of primaries are arranged mosaic-wise corresponding to said photoelectric converting elements; and 

interpolating means for generating a plurality of color signal components at a position shifted by half a pixel 
in horizontal and vertical directions from the center of an arbitrary pixel, from color signal components of a plurality 
of neighboring pixels. 
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FIG. 2 
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FIG. 3A 



FIG. 3B 



FIG. 3C 
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FIG. 5A 
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FIG. 9 
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FIG. 12A 
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FIG. 16 
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(54) 1-Chip color video camera for generating Inter-pixel color signal component by interpolating 
primary color signals from neighboring pixels 



(57) A 1 -chip color video camera providing such a 
frequency characteristic that suffers from small attenua- 
tion up to high frequency range for a G signal which has 
highest degree of contribution to brightness, and hence 
providing high resolution, is disclosed. In the 1-chip 
color video camera (100) having a color separation cir- 
cuit (8) for processing signals obtained from a solid- 
state image sensor (1) in which color filters (85) of R. G 
FIG- a 



and B primaries arranged mosaic- wise for respective 
pixels, color signal components at a central portion of a 
pixel block consisting of four pixels of two rows by two 
columns on the solid-state image sensor are generated 
by interpolation using color signal components of a plu- 
rality of neighboring pixels. 




EP 0 729 278 A3 



J) 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 96 IG 2856 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropfiate, 
of relevant passages 



Relevant 
to clainn 



CLASSIFICATION OF THE 
APPLICATION (lnl.CI.6) 



X 
A 

X 

A 



EP 0 483 837 A (SANYO ELECTRIC CO) 6 May 
1992 

* page 4, column 4, line 29 - page 7, 
column 10, line 54; figures 1A,1B * 

EP 0 630 159 A {MATSUSHITA ELECTRIC INO CO 
LTD) 21 December 1994 

* page 4, column 6, line 12 - page 5, 
column 7, line 23; figures 1.2,4,5 * 



1,2,6,7. 
11 
3,8 



1,2,6,7, 
11 
3.8 



HO4N9/04 



TECHNICAL FIELDS 
S6ARCHEO (lnt.CI.6) 



H04N 



The present searcli report has been drawn up for all daims 



Place o* *m*tzn 



THE HAGUE 



Oats ot complation at th« mwCh 

13 July 1998 



De Paepe, W 



CATEGORY OF CrTED DOCUMENTS 

X : pn/tieuUrty rvlrvant H taken alone 

Y : p«rboulorty rvlevant if oombin«d with another 

dooumttnt of the aame calegary 
A : techno log icAl background 
O : non-written ditclosure 
P : intermediate documant 



T : theory or prinople underlying the invention 
E : eortier patent dooumertt, but published on. or 

after the filing dote 
0 : documsnt oit«d in the epplioction 
L : document cited for other reoaon* 

& : member of the same patent family, corretpondtng 
document 



2 



